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About the research 


Prevalence and outcomes of workplace-based delivery in VET 


Kristen Osborne, NCVER 


Extensive evidence demonstrates a trend of positive outcomes related to learning in the workplace. 
Despite this, no examination of the outcomes has been undertaken using the available Australian 
vocational education and training (VET) data. 


This publication uses VET administrative data to examine trends in the delivery of workplace-based 
training, including where entire programs are delivered in this way. Additionally, data from the National 
Student Outcomes Survey are used to model the effect on employment and achievement outcomes of a 
student receiving workplace-based delivery. As previous work has extensively examined the role of 
workplace learning in apprenticeships and traineeships, this publication does not include these students. 


Key messages 


= Workplace-based delivery was used for about 4.1 million subjects (representing 17.2% of all subjects 
delivered outside an apprenticeship or traineeship) in 2019, either as the sole mode of delivery or in 
combination with other modes. 


« Around 800 000 students experienced workplace-based delivery as part of their VET journey in 2019, 
outside an apprenticeship or traineeship. 


«= The most notable predictor of a student receiving workplace-based delivery when available factors 
were modelled was the field of education of their study. 


« The relationship between mode of delivery and factors such as the field of education studied 
complicates efforts to understand the effects of workplace-based delivery. As a likely consequence of 
this, the analysis of the impact of workplace-based delivery using administrative data did not identify 
a material impact on student outcomes. 


" Amore conclusive understanding of the extent of the impact exerted by workplace-based delivery on 
student outcomes may only be possible through a randomised trial, whereby the only difference in the 
program is the presence of workplace-based delivery. 


Simon Walker 
Managing Director, NCVER 
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© Executive summary 


Learning on the job can be a valuable and rewarding element of a program of study, with a recent 
review of evidence in this area finding it to have positive effects on the outcomes of vocational 
education and training (VET) students (Osborne et al. 2020). The specific benefits of workplace-based 
education on the long-term outcomes of young people in training have also been identified (Waugh & 
Circelli 2021). This publication uses administrative data from the National VET Provider Collection to 
summarise the trends in workplace-based delivery of subjects for VET students, as well as analytic 
modelling to examine how different factors might predict this type of delivery. The modelling is also 
used to identify the factors that affect outcomes, and it attempts to quantify the positive effects of 
workplace-based delivery. 


Three modes of delivery are possible for each subject: internal, external and workplace-based or a 
mix of modes. To summarise the use of workplace-based delivery, subjects were split into three 
general categories: 


e those that were workplace-based 
e those that were workplace-based alongside other modes 
e those that were not workplace-based at all. 


In 2019, over 20% of all subjects were delivered with some degree of workplace-based training, 
representing more than 5.5 million individual subjects. Of these, just over 74% (or around 4.1 million) 
were not part of an apprenticeship or traineeship. Around 800 000 students experienced workplace- 
based delivery as part of their VET journey in 2019 outside an apprenticeship or traineeship. 


Many programs use workplace-based delivery for all of their subjects. For example, the Certificate III 
in Electric Passenger Train Guard and the Advanced Diploma of Competitive Systems each had more 
than 2000 student enrolments in 2019, all of which were delivered in the workplace. Other programs, 
such as the Certificate IV and Diploma of Ministry, used workplace-based delivery when combined with 
other delivery modes. Although the programs that were reliant on workplace-based delivery often had 
a structure intentionally similar to formal apprenticeships and traineeships, no contract of training 
was required. 


Investigation at the student level shows variation between the proportion of students receiving 
workplace-based delivery depending on their residential state or territory. For example, higher 
proportions of non-apprentice or non-trainee students in Tasmania (29.5%) and Queensland (21.3%) 
received some amount of workplace-based learning when compared with those in other states and 
territories. There was also significant variation in workplace-based delivery received between non- 
apprentice and non-trainee students with different study modes: 27.4% of full-time students experienced 
some amount of workplace-based delivery, compared with 19.9% of part-time students. Other student 
factors, such as gender and disability status, were also compared, but within these factors there was 
generally variation of fewer than three percentage points between the different categories. 


An analysis of the factors that predict workplace-based delivery reveal that students’ field of education 
is often the most relevant: 


e Fields such as radiography and pharmacy increase the likelihood of workplace-based delivery 
more than 20 times, compared with the benchmark field of business and management. 
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e Philosophy and religious studies, medical studies, forestry studies and justice and law 
enforcement are all more than five times more likely than the benchmark to include workplace- 
based delivery. 


e On the other hand, biological sciences, accountancy and behavioural science were all associated 
with a reduction in the likelihood of workplace-based delivery of four times or more, again 
compared with the benchmark. 


A further analysis focused on outcomes, specifically improvements in employment status following 
training and the achievement of a student’s main reason for training. The analysis compared students 
who had received any amount of workplace-based delivery with those who had not (once again excluding 
apprentices and trainees). The analysis demonstrated no conclusive impact on these outcomes resulting 
from the use of workplace-based delivery in a student’s training; however, the model could only account 
for a limited range of known factors and make a broad assessment of impacts. Nevertheless, previous 
research has found meaningful positive effects from learning in a workplace during training (Bahl & 
Dietzen 2019; Billett 2019; Kamaliah et al. 2018). Given the evidence supporting a positive impact on 
student outcomes from workplace-based delivery, the results from this empirical research should be 
viewed in the context of the limitations of administrative data. 


The lack of material differences in outcomes may be due to the interrelated nature of many of the 
factors used; for example, workplace-based delivery is often concentrated by field of education and 
level of education. When these factors are included in the analytic model and are considered, they may 
also account for the impact of workplace-based delivery and result in no material effect of the delivery 
mode. However, as factors such as field of education are also outcome predictors, it is important that 
these are included in outcome modelling. 


Furthermore, specific programs tend to include similar amounts of workplace-based delivery across all 
students, since the mode of delivery and/or assessment are routinely mandated by program rules. This 
means that for many programs there will be no ‘comparison groups’ of students - those who did and 
those who did not experience workplace-based delivery. While historical evidence supports the positive 
effects of learning in the workplace, ultimately, it may only be possible to distinguish the effects of 
workplace-based training on student outcomes through a randomised trial, one in which the only 
difference in the program is the presence of workplace-based delivery. This would provide more reliable 
data on the degree of effect of workplace-based delivery on student outcomes. 
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é& The state of workplace-based VET 


Before an analysis of the data is undertaken, it is Key points 

important to define the concept of workplace-based « Workplace-based delivery is VET training 
VET. Terms such as ‘work-based learning’, ‘work- conducted in the workplace. 

based education’ or ‘work-integrated learning’ are = In 2019, around 17% of all subjects 

often used when discussing education that takes delivered outside an apprenticeship or 
place in the workplace as part of VET programs. The traineeship were either partially or fully 
format of such programs might include work delivered in the workplace. 

placements or incorporate a student’s existing work. = More than one-fifth of these subjects were 
These programs might be referred to as delivered exclusively in the workplace in 
apprenticeships, traineeships or internships. In any 2019. 


case, the key element is the inclusion of learning in 

a workplace as part of the training program’s structure; that is, integrated into its educational approach. 
The reason for investigating the workplace-based delivery of subjects is simple: there is compelling 
evidence that work-based education in general can have strong positive effects for students. For a broad 
synthesis of this evidence, see Osborne et al. (2020), Work-based education in VET. 


The term ‘workplace-based VET’ is used here because it relates directly to how subjects are delivered to 
students. The Australian Vocational Education and Training Management Information Statistical Standard 
(AVETMISS), which covers the national VET data collections, classifies subject delivery mode in the 
following way: 


= internal delivery (for example, classroom-based), where the student and the trainer attend a 
training delivery location. This includes when the training is delivered using video or internet links 
in real time 


= external delivery (for example, online), where the student does not primarily attend a delivery 
location but instead undertakes training in their own time and location using training materials 
provided online or by correspondence. Contact with the trainer is usually limited to feedback on 
submitted work 


= workplace-based delivery, where the training is conducted in the workplace (irrespective of whether 
it is conducted by the training organisation or the employer). This includes industrial/work 
experience, field placement, fully on-the-job training or structured workplace training delivered at a 
place of employment (NCVER 2019). 


Training can use any combination of these modes. For example, a student who attends classes for a 
subject at their local TAFE (technical and further education) institute two days a week and has one day 
of work placement is experiencing a mixed internal and workplace-based delivery mode. A student 
studying an entirely online course from home would be experiencing external delivery. Finally, a worker 
studying on the job with an enterprise registered training organisation (ERTO) might train entirely on the 
job, resulting in exclusively workplace-based delivery. 


Just as subjects can be delivered using any combination of delivery modes, a program may similarly 
comprise a range of modes across subjects (and a student experiences a variety of modes). For this 
reason, this research analyses either the characteristics of particular subjects, or of the program of 
which the subject was part. 
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Patterns of workplace-based VET 


Any investigation of the outcomes of workplace-based training needs to begin with an understanding of 
the patterns in the use of workplace-based training itself. Any significant differences in location or other 
delivery characteristics will potentially translate into differences in outcomes. The trend analysis begins 
at the most granular - the subject level. It then moves to the program and student levels. 


Subject-level trends 


Figure 1 provides a brief overview of the volume and proportion of workplace-based delivery in 2019, 
including, in this instance, the extent of this type of delivery in apprenticeships and traineeships. 


Figure 1 Summary of all workplace-based training in 2019 


In 2019, there were 26.6 million 


subject enrolments 


About 5.5 million subjects had a 
workplace-based element 


20.6” 


ie) No workplace-based element (e) Workplace-based in part or whole 


Of these, 4.1 million were done outside of an 
apprenticeship or traineeship 


@ Not part of an apprenticeship or traineeship ® Part of an apprenticeship or traineeship 


Note: All figures exclude subjects with no valid delivery mode, such as recognition of prior learning or credit transfer. 
Source: National VET Provider Collection, 2019. 
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A more detailed breakdown by subject delivery mode is presented in Table 1. Excluding subjects delivered 
as part of an apprenticeship or traineeship, 17.2% of subjects included some amount of workplace-based 
delivery, while 4.1% of subjects supported only this type of delivery in 2019. This means that more than 
one-fifth of subjects with any workplace-based delivery were exclusively workplace-based (23.7%). 


Table 1 Subject enrolments by delivery mode, 2019 


Delivery mode N (7000) % 
Any workplace-based delivery 4 061.8 17.2 
Workplace-based only 963.6 4.1 
External and workplace-based 940.1 4.0 
Internal and workplace-based 880.2 3.7 
Internal, external and workplace-based 1277.9 5.4 
No workplace-based delivery 19 531.2 82.8 
External only 2 528.5 10.7 
Internal only 14 280.5 60.5 
Internal and external 2 722.2 11.5 
Total 23 593.0 100.0 


Notes: Excludes subjects delivered as part of an apprenticeship or traineeship, and those with no applicable 

delivery mode (such as subjects completed through recognition of prior learning or credit transfer). 
Source: National VET Provider Collection, 2019. 
When examining the modes of delivery that different types of training tend to use, two trends stand out. 
Subjects not part of a nationally recognised program had the highest rate of workplace-based delivery only 
(5.2%; figure 2, table 2). Figure 2 shows that training package qualifications contained the highest rate of 
any workplace-based delivery; that is, only workplace-based plus mixed workplace-based (18.2%). 


Figure 2 Subject enrolments by type of training and delivery mode, 2019 (%) 


Training package qualifications [Rx 14.3 81.8 
Training package skill sets [Za 11.4 84.1 
Accredited qualifications [Ex 96.0 
Accredited courses [@ReyKyy 91.9 


Subjects not part of a 


. : 52 12.3 82.5 
nationally recognised program 
0 10 20 30 40 50 60 70 80 90 100 
m= Workplace-based delivery only = Mixed workplace-based delivery = No workplace-based delivery 


Notes: Excludes subjects delivered as part of an apprenticeship or traineeship and those with no applicable 
delivery mode (such as subjects completed through recognition of prior learning or credit transfer). 


Source: National VET Provider Collection, 2019. 


A strong divide in the proportion of workplace-based delivery is seen between qualifications at 
certificate | and Il, and those at certificate III and above (figure 3): proportionally, many more subjects 
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had a workplace-based delivery element at certificate III and above. On average, 20.8% of subjects at 
certificate III and above included workplace-based delivery, compared with an average of 4.8% of 
subjects at certificate Il and below. 


Figure 3 Subject enrolments by level of associated qualification, 2019 (%) 


Diploma or higher [eg 13.1 83.7 
Certificate IV [Rae 18.1 77.7 
Certificate Ill [Za 19.7 76.1 
Certificate Il FKP 94.1 
Certificate | JAS 96.4 
0 10 20 30 40 50 60 70 80 90 100 
= Workplace-based delivery only = Mixed workplace-based delivery = No workplace-based delivery 


Notes: Excludes subjects delivered as part of an apprenticeship or traineeship, and those with no applicable 
delivery mode (such as subjects completed through recognition of prior learning or credit transfer). 


Source: National VET Provider Collection, 2019. 


When analysing the level of workplace-based delivery by the funding source of a subject, a clear trend 
emerges (figure 4). More than one-fifth (21.2%) of government-funded subjects were at least partially 
workplace-based, with 5.6% using only workplace-based delivery. This compares with 15% of domestic 
fee-for-service subjects, and only 6.3% of international fee-for-service subjects. 


Figure 4 Subject enrolments by funding source and delivery mode, 2019 (%) 


Government funded 


Domestic fee-for-service Km 


International fee-for-service 


0 10 20 30 40 50 60 70 80 90 100 
= Workplace-based delivery only = Mixed workplace-based delivery = No workplace-based delivery 
Notes: Excludes subjects delivered as part of an apprenticeship or traineeship, and those with no applicable 


delivery mode (such as subjects completed through recognition of prior learning or credit transfer). 
Source: National VET Provider Collection, 2019. 
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Program-level trends 


Delivery mode is observable at the subject level, with aggregation used to translate this toa 
program level. This was done by calculating the proportion of subjects associated with a program 
that supported some amount of workplace-based delivery. For example, a program in which half of 
the subject enrolments (across all program enrolments and students) were delivered in the 
workplace and half were delivered in the classroom would have a workplace-based delivery rate of 
50%. Given the much higher proportions of workplace-based delivery in training package 
qualifications and skill sets than in the other types of programs (figure 2), the analysis here is 
concentrated on these two types of training. 


Figure 5 shows the relationship between the numbers of training package qualifications and proportions 
of workplace-based learning as either a total or partial delivery mode in 2019. This gives a perspective 
on the distribution of workplace-based learning across different qualifications. 


Figure 5 Proportion of any workplace-based delivery within a training package qualification, 2019 


600 
500 
400 
300 
200 


100 


Number of training package qualifications 


0 10 20 30 40 50 60 70 80 90 100 
Proportion of workplace-based learning within training package qualification 


Notes: Only includes training package qualifications with one of more associated subject enrolments not part of an apprenticeship or 
traineeship, or with no applicable delivery mode (such as subjects completed through recognition of prior learning or credit 
transfer). N = 1464. 


Source: National VET Provider Collection, 2019. 


Of the 1464 training package qualifications counted in figure 5, 35.1% (514) included no workplace-based 
delivery at all in 2019. The distribution demonstrates that, when workplace-based delivery is included, it 
is most often either a small (<10) or large (>90) proportion of all subjects associated with that 
qualification in a year. This suggests an ‘all or nothing’ approach to the use of workplace-based delivery 
for this category of training. 


This divide is even more pronounced for training package skill sets (figure 6). Of 211 skill sets, 48.3% 
(102) had no workplace-based delivery in 2019, and 22.7% (48) had workplace-based delivery in every 
associated subject, either exclusively or mixed with other delivery modes. 
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Figure 6 Proportion of any workplace-based delivery within a training package skill set, 2019 
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0 10 20 30 40 50 60 70 80 90 100 
Proportion of workplace-based learning within training package qualification 


Notes: Only includes skill sets with one of more associated subject enrolments not part of an apprenticeship or traineeship, or with no 
applicable delivery mode (such as subjects completed through recognition of prior learning or credit transfer). N = 211. 


Source: National VET Provider Collection, 2020. 


To give a more detailed perspective on the programs relying on workplace-based delivery, table 2 
shows the most popular programs where all the subjects included some element of workplace-based 
delivery. Once again, this table excludes programs that were part of an apprenticeship or traineeship. 


A general trend in table 2 is the presence of programs that are normally undertaken by existing 
workers. For example, those studying the Certificate III in Electric Passenger Train Guard or the 
Certificate IV in Swimming Pool and Spa Building are likely to need workplace experience to learn and 
practise the relevant skills. Similarly, training package skill sets and accredited courses are 
appropriate for the upskilling of existing workers: it is unlikely that the opportunity and motivation 
will be present for a student to enrol in and complete the skill set ‘SIFSS00004 - Safe Gravedigging’ 
outside an existing work context. Additionally, training providers require ‘evidence of a relevant 
professional role’ for many of the health-related accredited courses, such as the Course in Ear and 
Hearing Health. 


The most popular accredited qualifications using workplace-based delivery in all associated subjects 
in 2019 were the Certificate IV, Diploma and Advanced Diploma of Ministry. These qualifications all 
emphasise workplace opportunities to apply knowledge and are normally undertaken in the context 
of a student’s role in a relevant religious organisation. In fact, these courses are similar in structure 
to a traineeship, but accredited qualifications are not typically offered as apprenticeships or 
traineeships. 
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Table 2. The five most populated programs in which all subjects included some element of workplace-based 
delivery by type of training, 2019 


Type of training (associated program) Workplace- Mixed Total 
based workplace- 
delivery only based delivery 


Training package qualifications 


TLI32315 - Certificate III in Electric Passenger Train Guard 2 953 0 2 953 
MSS60316 - Advanced Diploma of Competitive Systems and Practices 2 384 0 2 384 
TLI32318 - Certificate III in Electric Passenger Train Guard 1 940 0 1 940 
CPC40808 - Certificate IV in Swimming Pool and Spa Building 2 1477 1479 
LGA40404 - Certificate IV in Local Government (Operational Works) 399 399 798 
Training package skill sets 

AVISS00053 - Aerodrome Reporting Officer Skill Set 0 518 518 
SIFSS00004 - Safe Gravedigging 126 301 427 
SIRSS00012 - Community Pharmacy Dispensary 0 308 308 
AHCSS00052 - Pork Industry Stockperson Skill Set 18 245 263 
RIISS00036 - Underground Shotfiring - Metalliferous Skill Set 196 16 212 
Accredited qualifications 

10669NAT - Certificate IV in Ministry 0 11 137 11 137 
10670NAT - Diploma of Ministry 0 5 583 5 583 
10671NAT - Advanced Diploma of Ministry 0 2 673 2 673 
10797NAT - Graduate Certificate in Applied Pharmacy Practice 0 2 494 2 494 
10454NAT - Diploma of Nutrition and Dietetics for Personal Trainers 0 957 957 
Accredited courses 

10630NAT - Course in Ear and Hearing Health 0 2 682 2 682 
10754NAT - Course in Immunisation Practice in Primary Healthcare 0 760 760 
10762NAT - Course in Wound Closure 0 531 531 
10637NAT - Course in Tympanometry 0 323 323 


10532NAT - Course in Selection and Installation of Child Restraints 0 149 149 


Notes: Only includes programs with one of more associated subject enrolments not part of an apprenticeship or traineeship, or with no 
applicable delivery mode (such as subject completed through recognition of prior learning or credit transfer). 


Source: National VET Provider Collection, 2019. 


Overall, it seems that the programs that always use work-based delivery are largely designed to be 
undertaken in conjunction with work in the relevant industry. This includes many programs where the 
student is likely to have an established career (such as accredited courses requiring potential students to 
be existing employees in the health field). Note that the programs in table 2 exclude those being 
undertaken as part of an apprenticeship or traineeship, despite the amount of workplace-based delivery 
taking place. Interestingly, the delivery mode patterns of these programs are similar to an 
apprenticeship model in terms of the quantum of workplace-based training but lack a formal contract of 
training. Further work may investigate the structure and nature of these programs and how they differ 
from formal apprenticeships and traineeships. 


Student trends 


While understanding the use of workplace-based delivery at the subject and program levels is important, it is 
equally important to understand how this translates to the student training experience. For this element of 
the analysis, students were divided into those who had received no workplace delivery, those with a mixture 
of workplace-based and other modes of delivery, and those whose only delivery mode was workplace-based. 
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Table 3 compares the proportion of workplace-based delivery within the subjects studied by a student in 


2019 by various factors. 


Table 3. Student attributes by workplace-based delivery, 2019 (%) 


Attribute Workplace-based Mixed workplace- No workplace-based 
delivery only based delivery delivery 
Gender 
Female 3.7 17.4 78.9 
Male 3.9 14.6 81.5 
Not known 6.1 38.6 55.3 
Disability status 
Without a disability 3.7 18.1 78.2 
With a disability 2.4 16.8 80.8 
Not known 5.5 8.1 86.4 
Indigenous status 
Non-Indigenous 3.2 16.9 79.9 
Indigenous 2.5 19.0 78.6 
Not known 9.4 15.5 75.1 
Study mode 
Full-time 2.1 25.2 72.6 
Part-time 4.1 15.9 80.1 
State/territory of residence 
New South Wales 2.8 16.7 80.5 
Victoria 4.5 13.9 81.6 
Queensland 6.5 14.8 78.7 
South Australia 2.4 11.6 86.0 
Western Australia 3.1 15.6 81.3 
Tasmania 5.3 24.2 70.5 
Northern Territory 3.8 12.6 83.7 
Australian Capital Territory 2.9 12.1 84.9 
Offshore 1.8 9.0 89.2 
Not known 2.3 40.5 57.1 
Total (‘000) 149.9 649.1 3 061.7 


Notes: Students with no valid delivery mode and apprentices/trainees were excluded. N = 3 860 904. 
Source: National VET Provider Collection, 2019. 


The factor with the most variation in table 3 is the state or territory of student residence. States such 


as Queensland and Tasmania had the highest rates of students receiving workplace-based delivery only 


in 2019 (6.5% and 5.3%, respectively). Students from Tasmania and New South Wales experienced 


higher rates of mixed workplace-based delivery (24.2% and 16.7%, respectively), while students 


residing offshore or in South Australia had the lowest rates of workplace-based only delivery 


(1.8% and 2.4%, respectively). 
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What are students saying? 


One insight into the student perspective on workplace-based learning is available through the National 
Student Outcomes Survey, which asks students to provide suggestions for improvement. These 
verbatim comments were analysed for any combination of the following broad keywords that 
qualification completers used to reference work-based learning: 


e work placement 

e internship 

e apprentice or apprenticeship 
e trainee or traineeship 


e work experience. 
While not a definitive list, it provides insight into students’ experiences with work-based learning. 


Of approximately 55 800 valid comments, 1418 comments contained one or more of the selected 
keywords. Many of these (614) suggested either introducing some work-based learning or increasing the 
volume of work-based learning hours. Very few (19) suggested that the number of work placement 
hours should be reduced. 


Although these comments came from only a small subset of qualification completers who took part in 
the survey, it is telling that, when they make the effort to comment on work-based learning, they ask 
for more rather than less. 


Comments critical of the way their training provider had handled work-based learning were also 
received, but these generally asked for more support or better organisation rather than the removal of 
the work-based learning component of the course. 


Overall, these findings suggest that students see learning in the workplace as a valuable component of 
their training. 


Prevalence and outcomes of workplace-based delivery in VET NCVER | 16 


@ Predicting workplace-based delivery 


Key points 


Many of the factors relevant to workplace-based delivery are likely 
to also be related to each other. For example, students in Tasmania 
are more likely to enrol in certain qualifications and be more likely 


Many varied factors contribute to 
predicting whether a student 
receives some workplace-based 


to receive workplace-based delivery. Separating these dependencies ; : a 
. ; ; delivery during training. 
requires an analysis that accounts for many factors simultaneously. 


= Certain fields of education, such as 


To better understand which factors are particularly important in those related to health and social 
predicting workplace-based delivery for a student, a logistic care, were the strongest predictors 
regression model was fitted to the data. This allowed the factors of workplace-based delivery. 


to be modelled independently from one another. The model 
outputs are provided in the appendix. 


The following factors (predictors) were included in the regression model, which used data from the 2019 
National VET Provide Collection: major field of education 


= major level of education = student residential state/territory 
= type of training = highest previous education level 

= disability status = apprenticeship/traineeship status 
= Indigenous status = full-time/part-time study status 

= labour force status = socioeconomic status (IRSD) 

= gender = student remoteness region 


= age group 


The analysis is focused on the relationship between the selected factors and whether the student received 
any workplace-based delivery at the student level. This is represented by a binary variable, where 1 meant 
a student experienced some workplace-based delivery and 0 meant they had not. These results do not 
relate to the receipt of workplace-based delivery at the subject or program level. Every overall factor used 
in the model was determined to be statistically significant in predicting workplace-based learning, but this 
is likely to be due to the size of the dataset used (greater than 4.2 million students). 


A key output of the logistic analysis used is the odds ratio. The odds ratio is a statistic that measures the 
strength of the association between a factor and an outcome. Here, the association is between factors 
such as a student’s field of education or residential location and the outcome of workplace-based 
delivery being included in a student’s training. The odds ratio always involves a comparison with a 
‘paseline’ factor. For example, if individuals studying business were twice as likely as those studying 
creative writing to receive workplace-based delivery, business students would have an odds ratio of 2. If 
individuals studying business were half as likely to receive workplace-based delivery as those studying 
creative writing, business students would have an odds ratio of 0.5. Standard benchmarks for odds ratios 
have been used (Table 4; Ferguson 2009). 
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Table 4 Examples of odds ratio effect sizes 


Effect type Small effect Moderate effect Strong effect 


Positive effects (more likely) 2.0-2.9 
0.50-0.34 


Notes: Odds ratios of less than two are generally considered to be too small to have a materially significant effect. Categories are 
guides for interpretation only. 


Source: Ferguson (2009). 


Negative effect (less likely) 


Significance and odds ratios were calculated for all values of all factors. However, many showed very small 
associations between the value of the factor and the presence of workplace-based delivery or were for 
values relating to ‘missing’ or ‘other’ responses and are therefore not presented in the following tables. 


Of the factors used in the regression analysis, it was field of education that had the strongest effect on 
the likelihood of workplace-based delivery. All fields of education were compared with a selected 
benchmark - business and management. This field was chosen as it is a popular field, is broad in scope 
and includes a range of qualification levels. The fields of education that most reduced the likelihood of 
workplace-based delivery are presented in table 5. 


Table 5 Lowest 10 odds ratio for fields of education by comparison with business and management, 2019 


Field of education compared with business Odds ratio point 95% confidence limits 
and management estimate 


General education programmes 0.34 0.33 0.36 
Chemical sciences 0.35 0.15 0.79 
Electrical and electronic engineering and technology 0.36 0.35 0.37 


Notes: Excludes those which were not significant (P>0.05) in chi-square test of maximum likelihood estimates. 
Source: National VET Provider Collection, 2019. 


Studying biological sciences, accountancy, behavioural science and computer science all had a strong 
negative effect on the likelihood of students’ receiving workplace-based delivery, compared with 
business and management. Students in these fields are more than four times less likely to receive 
workplace-based learning than business and management students (as they have odds ratios of 0.25 or 
less). Table 5 shows a general trend of lower probabilities in the fields of education related to 
professional white-collar jobs such as accountant (in Accountancy in table 5) and programmer/network 
administrator (in Computer science in table 5). 


Table 6 shows the alternative: those fields of education associated with the highest relative likelihood of 
workplace-based delivery. Students in these fields are at least three times more likely than business and 
management students to receive some workplace-based delivery. 
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Table 6 Highest 10 odds ratio for fields of education by comparison with business and management, 2019 


Field of education compared with business Odds ratio point 95% confidence limits 
and management estimate 


Notes: Excludes those which were not significant (P>0.05) in chi-square test of maximum likelihood estimates. 
Source: National VET Provider Collection, 2020. 

Many of the fields in table 6 are in the health and social care area. This includes pharmacy, radiography, 
medical studies and human welfare studies. These areas routinely require competency to be assessed in 
either a workplace environment or simulated equivalent, and/or for students to complete a specified 
minimum number of hours of work placement. ' 


The results also show differences in the likelihood of workplace-based delivery across the states and 
territories (figure 7). Here the point of comparison was the chance of a student residing in New South 
Wales receiving some amount of workplace-based delivery by comparison with other states and 
territories. The black line in figure 7 indicates an odds of 1; that is, no increase or decrease in the 
likelihood of workplace-based delivery. 


Figure 7 Odds ratios for state/territories, compared with NSW, 2019 
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Notes: Error bars show 95% confidence limits. 
Source: National VET Provider Collection, 2019. 


1 For details of the structure of these programs, see <https://training.gov.au/Home/Tga>. 
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Figure 7 shows that residing in Tasmania or Queensland slightly increases the likelihood of workplace- 
based delivery relative to residing in New South Wales. Residing in New South Wales makes workplace- 
based delivery more likely than residing in any of the remaining other states and territories, although 
these effects are all very small. 


The analysis of the impact of different factors on the likelihood of workplace-based delivery is key to 
understanding outcomes. If students are likely to receive certain outcomes for the same reason(s) they 
are likely to have their training delivered in the workplace, these factor(s) need to be controlled. For 
example, these results clearly show that certain fields of study are strongly connected to workplace- 
based delivery, meaning that outcome analyses need to be controlled for the student’s field of 
education. An ideal scenario would be the random assignment of students studying the same program 
(and therefore the same field of education) to either undergoing workplace-based delivery or not. In the 
absence of this logistically difficult task, analytical techniques can be used to understand better the 
effects of workplace-based delivery on outcomes. 
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@ Outcomes from workplace-based delivery 
Key points 


The key question remains: does workplace-based 
Factors other than workplace-based delivery 


delivery improve student outcomes? As in the previous ames oo. 
: — . account for significantly more of the variation in 
section, a logistic model was fitted to the data to 
. . student outcomes. 
understand the relationship between the presence of 


= Analysis of the i t of kplace-based 
workplace-based delivery and specific target factors Pigeon eres Mareveney cen cape eet 


: delivery using administrative data did not 
that represent possible student outcomes. Model ; : a 
; . . identify a material impact on student outcomes. 
outputs are provided in the appendix and further 


model details are available on request. 


The following were used in each model as the predictive factors on data from the 2020 National Student 
Outcomes Survey: 


= major field of education 

= disability status 

= Indigenous status 

= labour force status before training 

= delivery mode (with or without workplace-based delivery) 
= apprenticeship/traineeship status 

= socioeconomic status (IRSD) 


= student remoteness region. 


Two models were run - one predicting improved employment status and one predicting whether a 
student had wholly or partially achieved their main reason for training. The aim was to account for the 
(known and documented) effects of factors such as socioeconomic status or field of education and 
isolate the effects of including workplace-based delivery. Once again, odds ratios were the main means 
of comparison between the effects of different factors (see page 16 for explanation of this measure). 


The presence of workplace-based delivery gave an odds ratio of 1.23, by comparison with no workplace- 
based delivery (table 7). This means, all other factors being equal, a student receiving workplace-based 
delivery was slightly more likely to have an improved employment status after training. However, a 
value this size is unlikely to have a material effect or impact. This value must be placed in the context 
of the predicted effects of other factors, seen in table 7. 


The business and management field of education was once again used as the comparison benchmark. 
With other factors being equal, students studying computer science were less likely to improve their 
employment status than those studying business and management. Those studying electrical and 
electronic engineering and technology were more likely to have an improved employment status than 
those studying business and management. 
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Table 7 Odds ratios, likelihood of improved employment post training, qualification completers, 2020 


Estimate factor Comparison factor Odds ratio 95% confidence 
point estimate limits 
Any workplace-based delivery No workplace-based delivery 1.23 1.20 1.26 
Apprenticeship/traineeship Not an apprenticeship or 2.46 2.36 2.56 
traineeship 

Field of education: Electrical and Field of education: Business 1.62 1.50 1.76 
electronic engineering and technology and management 

Field of education: Computer science Field of education: Business 0.40 0.34 0.47 


and management 


Not employed before training Employed before training 0.41 0.40 0.42 


Notes: Further detail of the model are available in tables A3 and A4 in the appendix. 

Source: National Student Outcomes Survey, 2020. 

The effects of other factors were included to underline the relatively minimal impact that workplace- 
based delivery had in this model. The presence of workplace-based learning in a student’s training 
exerted a very small positive effect on post-training employment status. A similar effect was seen when 
modelling to determine whether a student achieved their main reasons for training (table 8). 


Table 8 Odds ratios, likelihood of achieving main reason for training, qualification completers, 2020 


Estimate factor Comparison factor Odds ratio 95% confidence 
point estimate limits 
Any workplace-based delivery No workplace-based delivery 1.24 1.18 1.31 


Field of education: Electrical and Field of education: Business 0.74 0.64 0.85 


electronic engineering and technology and management 
Field of education: Computer science Field of education: Business 0.77 0.61 0.99 


and management 
Not employed before training Employed before training 0.66 0.63 0.68 


Note: Further details of the model are available in the tables A5 and A6 in the appendix. 
Source: Student Outcomes Survey, 2020. 


The effect of workplace-based delivery appears to have a very small positive effect (although it is still 
statistically significant). Materially, the results from this analysis indicate that the effect of workplace- 
based delivery is too small to be considered as a meaningful influence on whether a student achieves 
their main reason for training. 


These results do not mean there were no students who experienced a strong positive effect from the 
workplace-based delivery of a subject and who had a better employment outcome or achieved their 
reasons for training as a direct result; rather, these results only outline the predicted effects in the 
context of the data available and in cases where other factors are being controlled. 


One possible scenario to explain the results is that the effects of field of education are strongly 
interlinked with those of workplace-based delivery. The analysis in earlier sections (tables 5 and 6) 
showed that field of education was often the best predictor of workplace-based delivery, and it is 
possible that once the effect of field of education is removed, any effects of workplace-based delivery 
are also removed. Many cases of workplace-based delivery are either the norm within a field, or 
compulsory (education, health, hospitality, trade etc.). Given the limited number of instances of a 
program being delivered without any workplace-based delivery, it is impossible for the model to 
disentangle the effect with the available data. The same effect may also be occurring with other 
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variables that are both related to the provision of workplace-based delivery and to the outcome a 
student achieves. 


Ultimately, it may only be possible to distinguish the effects of workplace-based training on student 
outcomes through a randomised trial, one in which the only difference in the program is the presence of 
workplace-based delivery. This would provide more reliable data on the degree of effect of workplace- 
based delivery on student outcomes. 
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QO Appendix — Technical details 


The analysis presented in this report includes logistic regression modelling to better understand factors 
that affect the likelihood of: 


- a student receiving workplace-based delivery 
- improved employment following training (qualification completers) 
- achieving the main reason for training (qualification completers) 


Tables A1-A6 provide the model outputs of these logistic regression analyses. 


Table A1 Model information — logistic regression predicting workplace-based delivery, 2019 


Response variable Workplace-based delivery mode 
Response levels 1 = any workplace-based delivery 
0 = no workplace-based delivery 
Response profile 1 0 
975 127 3 197 995 
Model Binary logit 
Optimisation technique Fisher’s scoring 
Number of observations read 4173 122 
Number of observations used 4173 122 
Model convergence status Convergence criterion (GCONV = 1E-8) satisfied 


Testing global null hypothesis: 


Test Chi-square Degrees of freedom Pr > Chi-square 
Likelihood ratio 701 313.707 135 <.0001 
Score 709 590.424 135 <.0001 
Wald 549 421.769 135 <.0001 
Model fit statistics 

Criterion Intercept only Intercept and covariates 
AIC 4 537 613.9 3 836 570.2 
sc 4 537 627.2 3 838 371.4 
-2 LogL 4 537 611.9 3 836 298.2 

Association of predicted probabilities and observed responses (for odds ratios) 
Per cent concordant 75.6 Somers’ D 0.512 
Per cent discordant 24.4 Gamma 0.512 
Per cent tied 0 Tau-a 0.192 
Pairs 3.32E+12 c 0.756 
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Table A2 Type 3 analysis of effects — logistic regression predicting workplace-based delivery, 2019 


Effect Degrees of Wald Chi- Pr > Chi 
freedom square square 
Major field of education 69 153 926.20 <.0001 
Major level of education 9 5 173.80 <.0001 
Disability status 2 25 853.18 <.0001 
Indigenous status 2 331.41 <.0001 
Labour force status 3 10 080.51 <.0001 
Gender 2 14 684.99 <.0001 
Age group 12 13 746.36 <.0001 
Student residential state 8 17 837.38 <.0001 
Highest previous education level 13 15 312.47 <.0001 
Apprenticeship/traineeship status 1 86 695.71 <.0001 
Full-time/part-time study status 1 1 026.42 <.0001 
Socioeconomic status (IRSD) 5 7 203.20 <.0001 
Remoteness score 6 48 635.16 <.0001 
Type of training 2 2 863.82 <.0001 


Note: Type of training = Unknown was set to 0 as it is a linear combination of other variable. 


Table A3 


Model information — logistic regression predicting improved employment status, 2019 


Response variable 
Response levels 


Response profile 


Model 

Optimisation technique 
Number of observations read 
Number of observations used 
Number of strata 

Weight variable 

Finite population correction 
Model convergence status 


Improved employment status 


1 = improved employment status 
2 = no improved employment status 


1 2 


80 729 65 045 
Binary logit 

Fisher’s scoring 

150 773 

145 774 

6 

SOS survey weights 

Used 


Convergence criterion (GCONV = 1E-8) satisfied 


Testing global null hypothesis: 


Test F value Num DF Den DF Pr>F 
Likelihood ratio 360.28 95.9 915 13 992 488 <.0001 
Score 356.88 96 145 673 <.0001 
Wald 274.8 96 145 673 <.0001 


Model fit statistics 


Criterion Intercept only Intercept and covariates 
AIC 689 300.56 605 246.32 
sc 689 311.69 606 325.5 
-2 LogL 689 298.56 605 052.32 
Association of predicted probabilities and observed responses (for odds ratios) 
Per cent concordant 72.5 Somers’ D 0.453 
Per cent discordant 27.2 Gamma 0.454 
Per cent tied 0.3 Tau-a 0.2247 
Pairs 5 251017 805 c 0.727 


Table A4 Type 3 analysis of effects — logistic regression predicting improved employment status, 2019 
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Effect F Value Num DF Den DF Pr>F 
Delivery mode 249.28 1 145 768 <.0001 
Apprenticeship/traineeship status 993.3 2 145 767 <.0001 
Socioeconomic status (IRSD) 54.66 5 145 764 <.0001 
Remoteness score 107.34 5 145 764 <.0001 
Labour force status before training 3 304.76 2 145 767 <.0001 
Indigenous status 14.03 2 145 767 <.0001 
Disability status 391.04 2 145 767 <.0001 
Major field of education 90.58 65 145 704 <.0001 
Funding source 99.5 1 145 768 <.0001 
State/territory of residence 28.61 7 145 762 <.0001 
Qualification level 313.18 4 145 765 <.0001 


Table A5 Model information — logistic regression predicting achieved main reason for training, 2019 


Response variable 
Response levels 


Response profile 


Model 

Optimisation technique 
Number of observations read 
Number of observations used 
Number of strata 

Weight variable 

Finite population correction 
Model convergence status 


Achieved main reason for training 


1 = Did achieve main reason for training 
2 = Did not achieve main reason for training 


1 2 


132 795 10 196 

Binary logit 

Fisher’s scoring 

148 175 

142 991 

6 

SOS survey weights 

Used 

Convergence criterion (GCONV = 1E-8) satisfied 


Testing global null hypothesis: 


Test F Value Num DF Den DF Pr>F 

Likelihood ratio 67.25 95.9952 13725 872 <.0001 

Score 60.76 96 142 890 <.0001 

Wald 304.8 96 142 890 <.0001 
Model fit statistics 

Criterion Intercept only Intercept and covariates 

AIC 244 860.86 229 153.5 

dSC 244 871.97 230 231.06 

-2 Log L 244 858.86 228 959.5 

Association of predicted probabilities and observed responses (for odds ratios) 

Per cent concordant 68.5 Somers’ D 0.384 

Per cent discordant 30.1 Gamma 0.39 

Per cent tied 1.4 Tau-a 0.052 

Pairs 1 237 358980 c 0.692 
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Table A6 Type 3 analysis of effects — logistic regression predicting achieved main reason for training, 2019 


Effect F Value Num DF Den DF Pr > F 
Delivery mode 75.44 1 142 985 <.0001 
Apprenticeship/traineeship status 354.99 2 142 984 <.0001 
Socioeconomic status (IRSD) 16.76 5 142 981 <.0001 
Remoteness score 23.76 5 142 981 <.0001 
Labour force status before training 197.46 2 142 984 <.0001 
Indigenous status 0.39 2 142 984 0.679 
Disability status 143.84 2 142 984 <.0001 
Major field of education 381.18 65 142 921 <.0001 
Funding source 248.85 1 142 985 <.0001 
State/territory of residence 27.27 7 142 979 <.0001 
Qualification level 106.3 142 982 <.0001 
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